posted 06-19-2008 11:26 AM            

I would like to know what you all think

This is the information provided at the beginning of the survey.

"ASTM has received a request from ASTM Committee E 52 on Psychophysiological Detection of Deception (PDD) Examiner Testing to explore the possibility of initiating a Proficiency Testing Program for industry professionals engaged in Psychophysiological Detection of Deception (PDD) Testing.

The program is intended for those examiners who want verification that they meet ASTM E2000 Standard Guide for the Minimum Basic Education and Training of Individuals Involved n the Detection of Deception and E2064 Standard Guide for Minimum Continuing Education of Individuals Involved in PDD"

------------------
Ex scientia veritas

posted 06-19-2008 12:48 PM         

How many different organizations do we have to maintain certification for each year....CAPE....AAPP.....APA....NPA??? Now twice a year for ASTM? The cost is also a factor but I still think the problem will be getting everyone reading out of the same play book.

Don't get me wrong, I am an ASTM member (so is Grogan....could he complete the process??...You bet!) and I fully support the goals of ASTM. I just don't think were are there yet.

Ted

posted 06-19-2008 02:04 PM            

Jim

posted 06-19-2008 02:48 PM            

Dear Psychophysiological Detection of Deception (PDD) Examiner Professional:

SUBJECT: Proposed Proficiency Testing Program on Psychophysiological Detection of Deception (PDD) ExaminerTesting

ASTM has received a request from ASTM Committee E 52 on Psychophysiological Detection of Deception (PDD) Examiner Testing to explore the possibility of initiating a Proficiency Testing Program for industry professionals engaged in Psychophysiological Detection of Deception (PDD) Testing.

The program is intended for those examiners who want verification that they meet ASTM E2000 Standard Guide for the Minimum Basic Education and Training of Individuals Involved n the Detection of Deception and E2064 Standard Guide for Minimum Continuing Education of Individuals Involved in PDD.

A short survey is enclosed to help determine what level of PDD Examiner support exists and to obtain valuable technical information that can help structure a program that will meet the needs of those who would participate.. A minimum of 30 professionals must register for the PTP program before test sample preparation & distribution services, data form development, statistical programming and other important PTP components can be performed by ASTM prior to program launch.

The ASTM-PTP and a Committee E52 Program Management Team of key experts in this profession will carefully review information obtained from this survey. Therefore, it is extremely important that you submit your completed survey by July 18, 2008.

ABOUT THE PROGRAM

PROGRAM TITLE: ASTM Proficiency Testing Program (PTP) on Psychophysiological Detection of Deception (PDD) Examiner Testing

PURPOSE:

To provide a statistical quality assurance tool to improve and maintain a high level of performance among PDD examiners. The resulting data allows participating PDD Examiners to demonstrate their proficiency in test data analysis and compare their results with other PDD Examiners globally. This ASTM International PTP program will establish itself as the first PTP program within the field of PDD. Additionally, test results will allow participants to:

Demonstrate their competence in established analysis protocols;
Receive useful feedback regarding performance that may assist in professional improvement;
Obtain a confidential report of their performance that is benchmarked against their peers;
Track their improvement over time;
Verify their scoring abilities for regulatory purposes and court acceptance (e.g. 2324);
Demonstrate their performance capability to consumers.

PROGRAM TESTS:

Each participant will receive a CD via mail containing 100 single-issue polygraph examinations where ground truth is known. Examinations included on the CD will be provided in the form of a PDF file. Participants will numerically analyze data for each examination and make decisions for each of these cases. As a benchmark, examiners wishing to meet published standards (e.g. paired-testing) must make fewer than 20% inconclusive decisions, and their decisions excluding inconclusive must be at least 86%. A higher standard, that of evidentiary testing, also limits inconclusive to 20% or lower, with decisions on the remaining cases to be 90% or better. Additionally, ASTM will provide instructions and an electronic template to enter data via personal computer which can then be submitted electronically to ASTM for statistical processing. Participants will always have the capability to recall their test results but not change them after they are officially submitted to ASTM for processing.

ASTM will electronically provide a final statistical summary report to those participants who submitted test results within 25 business days from the data submission deadline date. Final reports shall contain:

Statistical analyses of test data (e.g. percentage of conclusive results and correct decisions, etc.);
Coded test results to maintain PTP participant confidentiality;
Charts plotting their test results versus other participants in the PTP program;
Other important graphs and information (e.g. analysis of errors in decision making)

Furthermore, any additional statistical analyses will be determined by the executive committee upon review of the established program. Future additions to the program may include a second set of 100 polygraph examinations specific to multiple-issue polygraph examinations.

PARTICIPANT BENFITS:

The proposed proficiency testing program will benefit recent graduates as well as experienced examiners in demonstrating their proficiency in Psychophysiological Detection of Deception (PDD) Examiner Testing. The program is intended for those examiners who want verification that they meet ASTM E2000 Standard Guide for the Minimum Basic Education and Training of Individuals Involved n the Detection of Deception and E2064 Standard Guide for Minimum Continuing Education of Individuals Involved in PDD.

WHO SHOULD PARTICIPATE:

Individual PDD examiners in private practice, employed with state or federal government and those located internationally. Examiners from all sectors of this testing community are welcome to participate.

STANDARDS TO BE INCLUDED IN PTP:

Initial development of the ASTM-PTP on PDD Examiner Testing will include the following E-52 standards:

1. Standard Practice for Quality Control of Psychophysiological Detection of Deception (Polygraph) Examinations: E 2031-99 (Reapproved 2004);

2. Standard Practices for Interpretation of Psychophysiological Detection of Deception (Polygraph) Data: E 2229-02;

3. Standard Guide for PDD Paired Testing: E 2324-04

The following standards may be included in the PTP scope at a later date:

1. Standard Guide for Minimum Basic Education and Training of Individuals Involved in the Detection of Deception: E 2000-05;

2. Standard Guide for PDD Examination Standards of Practice: E 2060-00;

3. Standard Guide for Minimum Continuing Education of Individuals Involved in Psychophysiological Detection of Deception: E 2064-00;

4. Standard Guide for Clinical Psychophysiological Detection of Deception Examinations for Sex Offenders: E 2080-06

5. Standard Guide for Minimum Training Requirements for Examiners Conducting Psychophysiological Detection of Deception Testing of Sex Offenders in Treatment, Probation or Other Similar Programs: E 2163-06;

6. Standard Guide for the Conduct of PDD Screening Examinations: E 2386-04;

PROGRAM SCHEDULE:

The Proficiency Testing Program for PDD Examiners will be conducted biannually with testing in June and December. All materials, including test instructions, and data report forms are made available to each participant at the beginning of each test trial. Participants will have approximately four weeks from the test distribution date to submit their final results. All results will be statistically analyzed in accord with scientific standards of statistical significance. A final summary report will be distributed electronically to participants in approximately 25 business days from the data submission deadlines.

DISTRIBUTION OF SAMPLES:

CDs containing test data will be packaged and distributed by ASTM to each participant via standard mail prior to each test trial. All test samples will be acquired from a reliable source having a database of ground truth examinations.

REGISTRATION FEE:

A reasonable fee for an annual subscription to this ASTM will be assigned to cover the cost of test materials, data analysis, report production and program administration costs prior to marketplace promotion. Until all program costs are ascertained ASTM a fee is not known as of this mailing.

Thank you for completing and returning the enclosed survey by JULY 18, 2008.

Sincerely,

Anne M. McKlindon

ASTM Proficiency Testing Programs

SURVEY

ASTM Proficiency Testing Program on
Psychophysiological Detection of Deception (PDD) ExaminerTesting

1. What is your OVERALL impression of the proposed ASTM Proficiency Testing Program as described above? Please be as specific as possible.

2. Do you feel this Proficiency Testing Program is of value and benefit to your profession?

If so, in what way?

If of no value or benefit, why not?

Please check appropriate response below.

3. How important is it for you as a PDD Examiner Professional to demonstrate your competence in established analysis protocols to those in your profession or consumers of your service?

___Very Important ____Somewhat Important ____ Not Important

4. How important is it for you as a PDD Examiner Professional to receive useful feedback regarding your performance that may assist in professional improvement and/or advancement?

___Very Important ____Somewhat Important ____ Not Important

5. How important is it for you as a PDD Examiner Professional to obtain a confidential report of your performance that is benchmarked against your peers?

___Very Important ____Somewhat Important ____ Not Important

6. How important is it for you as a PDD Examiner Professional to have the ability to track and defend your professional performance and/or improvement over time?

___Very Important ____Somewhat Important ____ Not Important

7. How important is it for you as a PDD Examiner Professional to have the ability to verify your scoring abilities for regulatory purposes and court acceptance (e.g. 2324)?

___Very Important ____Somewhat Important ____ Not Important

8. How important is it for you as a PDD Examiner Professional to have the ability to demonstrate their performance capability to consumers of your professional service?

___Very Important ____Somewhat Important ____ Not Important

9. What would you add or subtract from this proposed PTP program to make it more attractive to participation in your profession?

10. Would you participate in this program twice a year? (Price will be REASONABLE)

____ Yes _____ No _____ Maybe

If no, could you PLEASE provide a reason why?

11. Additional comments:

Thank you for your response to this survey BY July 18, 2008

Providing your name and address below will ensure you receive future updates on this PROPOSED ASTM

PTP on Psychophysiological Detection of Deception (PDD) Examiner Testing

ASTM-PTP

Attention: Anne McKlindon

100 Barr Harbor Drive

West Conshohocken, PA 19428 2959

fax: 610-832-9668 OR

e-mail: PTP@ASTM.ORG

Name: __________________________________________________

Organization: _____________________________________________

Address: _________________________________________________

City : ___________________________________State: _________

Country: ______________________Zip Code: ___________________

Phone: _______________________ E-mail:______________

posted 06-19-2008 05:05 PM            

The return email is PTP@astm.org I guess this stands for Proficiency Testing Program. and someone set up the email to give the impression that the program already exists. I'm thinking that only ASTM members are receiving the survey at this time. I have been an E-52 member but my dues are currently in arrears. so I guess I'm not quite official.

Without going into my specific responses to the survey, in general, I do not think that ASTM should be involved in proficiency testing of PDD examiners IN ANY WAY.

Polygraph professional organizations and state licensing boards are better suited to insure examiner proficiency. ASTM does not test Doctors, Lawyers, or Nuclear Physicists; they shouldn’t have any oversight of PDD efficiency. (as a side note if YOU think that as a polygraph examiner you are uniquely qualified and want to help write standards for Nuclear Reactors, all you have to do is pay the ASTM Fee and you're IN.

In addition, I found their survey questions a bit presumptive and leading.

{I just deleted a lengthy rant about why I don't think routine admission of polygraph results is a good idea because my opinion didn't really have much to do with examiner proficiency..another time and place perhaps.}

If we are going to have proficiency testing, and this is going to take volumes of "Cussin' and Discussin', I really don't think ASTM is the proper venue.

------------------
Ex scientia veritas

posted 06-19-2008 06:15 PM            

I think only committee chairs saw this in the early stages. It's now out there for input to see if it's something worth pursuing. I think it is because right now, there is no meaningful certification out there - unless you go to Massachusetts and get tested by the court system, which they farm out to a university for a lab study. In this instance, we have much more control.

posted 06-19-2008 06:24 PM            

quote:

---

as a side note if YOU think that as a polygraph examiner you are uniquely qualified and want to help write standards for Nuclear Reactors, all you have to do is pay the ASTM Fee and you're IN

---

Let me address this one separately. This is what give ASTM the respect they have - and it's the reason they are often become law in many states. (Just search "ASTM" in your local statutes and codes.)

That doesn't mean you're going to have any influence in a committee. You would have to write a standard all those professionals would sign on to. If you object to a standard, you'd have to articulate why and convince the nuclear scientists they were wrong. Good luck. In other words, you'd be wasting your time, which is why Grogan even knows to keep quiet because it's an exercise in futility.

Your membership in the committee wouldn't be the same as that of the scientists. You'd have another class, and ASTM can control how many yahoos get in to keep the committee balanced to avoid a takeover by non-nuclear reactor scientists - or whoever makes up your hypothetical committee.

So, in the end, you're way off in your understanding of how ASTM works and what it does. You can oppose the idea, but you need to be careful not to mislead others who also don't know how it works.

posted 06-19-2008 07:21 PM            

As to your comment about this proficiency testing being "completely voluntary" I think you fail to acknowledge the impact of the backstabbing that some examiners use to criticize each other. This will likely result in the sort of backhanded extortion that occurs the first time an ASTM certified examiner belittles his competitors as less than competent because they don't have the ASTM seal of approval.

People who participate in this proficiency testing will likely start adding the ASTM seal to their advertisement and in order to compete with the appearance of proficiency or professionalism associated with the seal, an examiner will have to reach into his back pocket and lay out some cash. I believe your comment was “if you're any good, having ASTM behind you will take you places”. The barely concealed inverse presumption being, if you don’t have the ASTM certification, you’re not any good.
That is not voluntary participation it is economic coercion.

Whatever money ASTM chooses to charge to oversee this proficiency testing would probably be better spent on continuing education.

------------------
Ex scientia veritas

posted 06-19-2008 07:42 PM            

Backstabbing? You don't think that'll happen anyhow?

Let's face it. Right now we have little to no idea of how well any of us test. We know about half of us won't meet the NAS's cut. Are you one of them? Am I?

Here's a link to ASTM's proficiency testing:
http://www.astm.org/STATQA/index.html

posted 06-20-2008 12:46 AM            

Since he currently meets ASTMs requirements as an examiner for committee membership is it safe to assume that he wouldn't be able to pass their proficiency testing?

I don't think it is fair for you to state that there are no standards for polygraph examiners. I successfully completed an accredited polygraph training program, completed the internship required by both my school and my states licensing board,I passed my test for licensing and attend at least the bare minimum continuing education required to maintain my license (usually 5 or 6 times the minimum requirement, but that is neither here nor there). All of those things are standards.

Your argument is, more accurately, that these standards are insufficient. I tend to agree with you, but I don't agree that ASTM is the appropriate forum for ensuring examiner proficiency through the successful completion of a written examination. I don't really see how you can objectively measure the quality of a pre-test or post-test interview on a written exam.

Think about all of the scrutiny a new examiner is subjected to during initial training. If written test scores were predictive of examiner quality then you should be able to rate examiners by virtue of their academic standing at the conclusion of training. The guy or gal who finished #1 in his/her class would be a better examiner than the person who finished #2. I would like to think that some would agree that this isn’t necessarily so.

On the other hand, if you want to have a realistic picture of an examiner’s proficiency then you should either subject him to post examination PEER review or devise a method of scenario testing that could be PEER rated.

No matter how consistent you can become with objective data collection, question formulation and scoring protocols, I have some serious doubts that you can objectively measure the process of pre-test and post test interview. Therein lays a very large portion of what makes and examiner good or bad at his job.

Even if an examiner passed whatever requirement we devise, there is no guarantee that his next examination will contain any of the proficiencies he demonstrated to pass the test.

Bi-annual proficiency testing for a polygraph examiner is very similar to an officer who goes to the firing range twice a year to demonstrate his skill with his sidearm. If this officer shoots somebody, his scores on the range have very little relevance. If he is the best marksman since Carlos Hathcock and shoots the wrong person, or shoots without proper justification, he is screwed. His marksmanship medals are just so much tin. If he proves on the range that he couldn't hit the ground with his hat and it is determined that he justifiably shot the correct person he'll probably be OK. Unlike a pre-test interview, the location of holes in a target can be objectively measured with a great deal of accuracy.

I think that standards for the way we do our work are desperately needed. At the present time ASTM seems to be the most practical venue for the polygraph community to enumerate those standards. One of my main problems with ASTM is that an examiner who is not a member of ASTM cannot view the standards without becoming a member or buying them from ASTM. Any examiner unwilling to write them a check has ZERO opportunity to even attempt to conform to their standards. You really shouldn’t have to pay for a copy of the rules you are expected to follow, but with this proposal you would not only have to pay for a copy of the rules, you will have to pay for the opportunity to prove you read them.

I think that the best possible use of ASTM PDD standards is probably as a guide for the review of an examiners work.

As I said before, passing their proficiency exam will do very little to insure that an examiners next test will conform to standards no matter how useful those same standards might be at evaluating an examiners last test.

------------------
Ex scientia veritas

posted 06-20-2008 01:41 AM         

"Certification" in many professions only involves paying a fee to a specific organization. If we, as a profession, are to rise above that, then we need to clean up our act. I think we are on the right track here, we just need to agree on how to proceed.

Ted

posted 06-20-2008 06:27 AM            

quote:

---

Your argument is, more accurately, that these standards are insufficient.

---

posted 06-20-2008 08:27 AM            

Do we need it? You tell me. Don Krapohl and I looked at the scores of those who tested for paired-testing certification (essentially the same testing being discussed here - and nobody complained about that). Many of those people were recruited because their peers said these guys were the best. The results? A lot of people didn't do so well.

We still have people going through "grueling" schools who don't learn validated techniques or scoring rules, so their peers only know how to validate substandard practices. That doesn't help any of us. We have licensing here in Maine. Others have it elsewhere. How many states require proficiency tests after the license is granted? How many guys with licenses couldn't get through the process today? Licensing is a start, but we all know lousy licensed examiners.

Who should do the testing? Should, for example, the APA? The APA's Marston school "investigation" was leaked to the AP site. Do you want the leaker (whoever it is) or a leaker to have access to a list of proficiency scores? I don't think you'll see that as a problem with ASTM. Don't get me wrong, I think the APA can do it, but ASTM has standards and means in place right now - and they are independent meaning non-examiners (e.g., consumers, scientists, judicial officers, etc) have input should they desire to get involved. With APA, we're looking at 2010 as stated in our Strategic Plan:

Benchmarks

1. By 2010, the Board shall create one competency-based certification process (i.e., evidentiary testing, paired-testing, investigative testing, applicant testing, immigration and asylum testing, PCSOT) modeled on those of other professions.

2. By 2011, the Board shall create one competency-based certification process modeled on those of other professions.

Personally, I think the cost from ASTM will be too high for any of us to do it, but I like the idea. If you have a proficeincy test from them, then you're going to be more respected in the courtroom where we are seeing more tests end up, and the courts know what ASTM is.

We could approach the American Bar Association to conduct the independent proficiency tests - if people want them. There are probably others too. Right now, is it all that meaningful to say you're a member of a national organization as far as competence is concerned. Just go to one of Elmer Criswell's QC classes and look at what our members are doing out there in the field - and they all make the same arguments as here.

What does continuing ed really mean? Do you have to pass a test saying you understood the presented material? No, so you don't even have to pay attention. You can run a thousand tests and attend every seminar in town, but you don't have to have the ability to apply any of it to add that all to your resume.

I just think we need to work to separate the men from the boys to get rid of the Grogans of the world. An objective test would help.

I tested for Marin certification, and I've got to tell you, 25 days to get that stuff back means you're going to be working a lot of extra time to get those tests scored. It's not easy. I'd estimate it takes 40 hours or so to score 100 charts in PDF format. I don't think that's realistic, but I still like the idea.

posted 06-20-2008 09:18 AM            

ASTM sets the professional standards for professions and manufacturing, it does not certify any examiner or producer has met them. That IS for the courts to decide.
Neither will someone taking an unproctored test issued by the ASTM and who "meets" the so-called minimal requirements be automatically accepted in a court of law.

This means anyone attempting to present ASTM credentials as an examiner will most certainly be attacked by the opposition and challenged at some level. Any psychologist or forensic specialist can attest to that.
What other profession is being "tested" by ASTM and receiving a "certification"?

As I have stated, I do not believe it has it in it's charter to conduct such testing and should never attempt to institute such an endeavour.

posted 06-20-2008 11:13 AM            

1. His/her ability to conduct a proper polygraph test;

2. Do a proper pre-test;

3. Design proper relevant questions based upon a review of case facts;

4. Develop meaningful comparison questions based upon an interview with an examinee about his past and background and make them meaningful to the examinee;

5. Select a proper format which will properly encompass the relevant issues to be tested;

6. Conduct an ethical, legally sufficient examination;

7. Write a comprehensive, technically sound and understandable report of the examination.

Testing one’s ability to Read charts as a means of determining their proficiency is about as logical as a cardiac surgeon or brain surgeon being certified based upon his ability to read an x-ray or CT scan.

I’m about fed up with ASTM trying to turn the polygraph examination into a group of standards like we are a paint can or a ladder. Hell, we use a computer and we plug it into the electrical outlet, let’s get Microsoft and underwriter’s laboratory to certify us. We talk to people, we could use the Columbia School of Broadcasting or the Dale Carnegie folks to certify we speak well.

I realize I’m in the minority when it comes to ASTM, but I’ve said from the start of our association with ASTM that I think in our desire to be accepted by somebody official, we tied our wagon to the wrong horse. Can someone tell me more about what ASTM standards and proficiency tests exist for clinical psychologists or for paralegals or ophthalmologists or finger print examiners or hand writing examiners??????

posted 06-20-2008 11:13 AM            

Even if a committee could devise a way to objectively measure polygraph proficiency via a written test, it stiil wouldn't be like the other testing, because ASTM has the ability to blindly change the sample content to insure fairness. What of the committee that writes the questions and answers for polygraph proficiency testing? Would they to be allowed to take their own test? Would they be allowed to generate income by conducting ASTM Polygraph Proficiency Pesting Seminars?
If so, will they have to pass another proficiency test to prove they are qualified to teach the proficiency curriculum? Who would write that test?

I also noticed: ASTM proficiency testing starts at around $300.00 and goes UP.

Do you have an example of any type of proficiency testing currently available from ASTM that is more directly related to what the committee is trying to do with polygraph examiners?

------------------
Ex scientia veritas

posted 06-20-2008 11:29 AM            

I applaud the work and dedication of those willing to work with ASTM to construct standards for our profession. Unfortunately committee consensus does not constitute a consensus of the polygraph profession. There is nothing in place to provide the community to approve or deny standards and ASTM is more than willing to allow the few to set the standards for the many.

------------------
Ex scientia veritas

posted 06-20-2008 11:47 AM            

Later when he had a better understanding of the process asked several members what they thought. He got enough positive info to talk to ASTM about the program and if the possiblity existed that they could do the same type of program they do for others belonging to ASTM.

This survey is an attempt by ASTM to find out what the interest level they would get from examiners. Because they have no idea where to go to find examiners they logically sent the survey to members for an initial response. They are essentially asking for our input on would it be something we would want and how should it be handled. They have only a basic idea given them from some members as a starting point.

They are not asking for you to critique them or their program. If we don't want it they could care less. Of course it will cost money. How else do you suppose they would do it? Remeber this is a non-profit organization. They pay the bills by selling the standards to non-members and charging a fee to belong to the membership, for which you get the standards free each year.

If you want to respond to the survey do so. If you don't want to don't but if you do how about being professional about it.

One other thing, at this time I believe they are only talking about test data analysis proficiency testing. It would not be quite the same as paired testing proficiency.

All this program would do for you is to say that you demonsrtated you could read charts with a certain degree of proficiency. Does that benifit you in anyway? I think it would but you have to decide for yourself. Astm does not enforce the standards we write and could not be expected to. They don't have to. Courts all over the world understand the need for standards and recognize ASTM as the leader in this area. You may not want to belong, you may not believe but if you go to court and get asked if you used ASTM standards and you didn't you will get a large surprise.

I love this forum even though I can't spend much time here any more. I believe we all have the right to voice our opinion but some people posting here need to go back and read the survey. They are asking for your ideas on this program. They are not being arrogant, they are not trying to make money, just not lose any. They have no dog in this fight which is why they are a great place to try to put these things together.

Even if they start a program, which actually means you and I, they don't care if you choose to participate or not. They just need enough to cover expenses.

Jack

posted 06-20-2008 12:23 PM            

Those issues (pre-test practices, etc) would have to be tested through another means as they are with paired testing (two phases, scoring and exam competence). This is only one facet, and I don't know if ASTM could even handle the rest. You are correct in that more must be tested. We all agree. If this can't be done, then forget the rest, which is much more complicated.

Jack is correct. They want to know if the idea (which came from polygraph examiners) is worth pursuing. Now that the idea is out there, how good does it look for us to say we want nothing to do with our competence being tested? ASTM may be the wrong venue, but answer the survey honestly and tell them what you think.

posted 06-20-2008 01:33 PM            

It clearly states the program is intended for those examiners who want verification that they meet ASTM E2000 Standard Guide for the Minimum Basic Education and Training of Individuals Involved n the Detection of Deception and E2064 Standard Guide for Minimum Continuing Education of Individuals Involved in PDD. It is obvious to me that Test data analysis is just "phase I"

I see nothing in the comments by either side of the issue that damage the substance of this discussion. I think it is a healthy process to extend the issue, project and discuss potential problems and pitfalls. We need to discuss not only "Where it is" and "where it will be tomorrow" but "where it may be going" ten years from now. If we determine that in ten steps we will be walking off a cliff, we need to decide when and where to turn or not take the first step at all.

I see the purpose of this forum as a place to take the gloves off and say what we think with out having to worry to much about being nice. Except of course we shouldn't make personal attacks on each other.

The language and comments that I intend to place in my survey response will be as professional as I can make them; however, I intend to let this discusson run its course to help me formulate my final response.

If anyone takes personal offense to anything I say here, please let me know. I'll review my statements and make any apologies I deem appropriate. Shucks I even apologised to GM on time and I have absolutely no respect for what he stands for.

------------------
Ex scientia veritas

posted 06-20-2008 01:34 PM            

1. A written examination testing knowledge of physiology, psychology, test data analysis, history, ethics. law and operations. Schools do it now. The APA could do it as a condition of membership and it could be done regionally several times per year.

2. Written scenarios that require the student/applicant to provide proper relevant/comparison questions and proper test format.

3. Review of operations through a review of 5-10 examination chart sets with the report and video/audo tapes if available.

4. For those who can't release or don't have video/audio tapes, a mock crime scenario pre-test conducted before a panel of observers or conducted on someone who is the proctor done at national and regional APA seminars.

5. The conduct of one test chart showing competency with operations and the instrument.

posted 06-20-2008 02:58 PM            

I always hoped for a testing process above our academy certificate.

I thought the "veterans" would have jumped all over my ass if I brought this up before.

200 tests is kind of the Cert I guess we all strive for. For a criminal examiner(in a state where pre employment is illegal), that is alot of tests. I know most of you guys have thousands. The examiner in the next county over went trough the academy in the late 80's and he only has 187 tests. I don't think the 200 tests should be the sole reason for certification.

posted 06-20-2008 03:53 PM         

I like your post and stole the following from it:

1. His/her ability to conduct a proper polygraph test;

2. Do a proper pre-test;

3. Design proper relevant questions based upon a review of case facts;

4. Develop meaningful comparison questions based upon an interview with an examinee about his past and background and make them meaningful to the examinee;

5. Select a proper format which will properly encompass the relevant issues to be tested;

6. Conduct an ethical, legally sufficient examination;

7. Write a comprehensive, technically sound and understandable report of the examination.

Unless you are a graduate of John Grogan's Polygraph Institute, I beleive everything Skip stated above, was something you had to do to get the hell out of polygraph school!

I know each of my skills were tested nunmerous times prior to graduating from Backster's. I am not opposed to taking an exam on an annual basis to prove I have maintained my skills. In fact, I think it is an excellent idea.

I would rather have a root canal and a colonostopy performed at than same time as opposed to having to score 100 charts, in 25 days, twice a year.

posted 06-21-2008 10:17 AM            

1) it is not wise to start writing ASTM standards simply because of Grogan. ASTM lacks the leverage to enforce standars.

2) their are three professional/ethical roles to be concerned about: a) setting standards for practice, training, etc., b) reviewing and certifying the competency of individual professionals, and c) enforcement or responding to problems.

I think we are hoping for (c), when in fact ASTM can serve only roles (a) and (b). I believe these roles (a and b) should be completely separate, as in separate organizations. We sometimes fail to apppreciate this, because we are all used to agency work, in which an agency services all functions: standard setting, certification, and enforcement. APA and our state organizations seem to me to forget the difference between an association and an agency. Agencies have a different kind of authority over their membership/employees. Associations are primarily cooperative. It is the local situation that holds the leverage for enforcement in private practice - as it should be. That is why therapists are licenced/enforced by the state, though they are certified by a national organization according to standards set by the professional/accrediting organization.

The result of crossing and mixing these roles (i.e., one organization sets standards, licenses others, and responds to complaints) is the concentration of an inappropriate ammount of authority and power in a single body or individual. If those individuals are also competing in the free-market business environment, we have a huge risk that the professional process degrade into a form of market protection, in which forceful personality seems to dominate the standards, referrals, certifications, training, and everything.

What we need is a healthy and reasonable dialog about the state and progress of polygraph science, and the value of maintaining ethical role boundaries in our professional associations and certification bodies.

3) Their are additional problems with overusing that same set of 100 cases - which is not properly constructed for the purpose of examiner certification (if is the one I'm thinking of). It is better for research. "Ecological validity" refers to how well our research experiments approximates live field situations. Toward that end, it is OK to leave in all the messy, CM,artifacted, FP, and FN cases. Research questions pertain to inferences about how most people/examiners would perform with a given set of problems. Certification questions pertain to a much narrower focus - can this examiner do what he is trained and expected to do. Scoring error (FP, FN) cases does not accomplish that, and actually confuses the issue. Error cases are a fact of real life. So, ecological validity concerns tell us to leave them in researchers samples. Leaving error items in certification samples/tests is like adding a power-lifting challenge to the proficiency exams for engineering students. It doesn't tell us abou their engineering skill, because it is not correlated. In fact, error cases might be expected to be negatively correlated with scoring proficiency, just as power-lifting might be negatively correlated with calculus or engineering skill. Sure, there are exceptions, and the average of all that is a research concern. Concern about individual proficiency requires much more than scoring a sample that is not properly constructed for certification purposes.

4) There is also a tendency for us to miss the point around certification and proficiency as a form of validity. It its worst form, competency judgments are based on social charm and charisma, but in real-life trench-work situations, arguments are generally settled by whom has the most forceful personality. The smarter way would be to pay attention to the details and facts of the argument.

The problem to remain aware of is that humans tend to respond to human factor like charisma, forceful personalities, strong opinions, individual loyalty, imprinting, and economic acce$$/marketing - or worse: shiney suits, dark glasses, gold chains, expensive shoes, and big watches. None of these things really have anything to do with individual competency, or the validity of the ideas we endorse.

Certification can become a sublter and more troublesome form of these same old problems, if we miss the real point.

The real point is: certification is only a starting point for making assumptions or judgements about the adequacy of a single examination. Unfortunately it is also too often the ending point. The real concern is the individual examination - how accurate is the one we're working on right now, because decisions affect individual persons, not just trends, and research samples.

This sometimes becomes very clear to courts, who are not interested in the behavioral patterns of the 1000s of persons who were used to norm the MMPI. The court is interested in the individual for whom they have to make a decision that will affect his or her future, not the normative sample.

In the case of polygraph testing, I believe we should be primarily interested in the merits of individual examinations that may become the basis of decisions that affect the futures of individuals.

Another problem with biannual recertification is access to samples. There are known practice and retest effects when rescoring the same data over and over. For this reason, a competent program will require continual access to new data. I doubt we or ASTM can sustain continuous access to new confirmed samples of data for each certification event. This is an easy task for chemical analysis of metals, because they simply find another organization (separate from ASTM) to provide known samples for the participating laboratories to test.

A look at webpage suggests the ASTM PTP is for laboratories. Regardless of what we call our testing suites, the kind of polygraph scoring which we seem to be discussing is about individual scoring competency, not a laboratory or overall program competency.

To say that someone scored a sample at 86% accurate or 95% accurate still doesn't address the concern about the validity of the examination.

It is very important that we work on these things. However, it is equally important that we not work too quickly. Things tend to become set in stone in the polygraph profession, regardless of whether they are based in science or sound theory.

Polygraph examiners believe in their training and their trainers, which is why we have to insist they not simply make things up. The most recent one I've begun to hear about is this bid'ness of using probation questions as CQs on PCSOT maintenance exams and treatment questions s CQs on monitoring exams - if they pass the maintenance questions, they fail the treatment questions, and if they pass the monitoring question they fail the probation questions... To me this sounds like the everlasting-gobstopper of private practice polygraph referrals, and likely to be regarded by our detractors as a justification for accelerating anti-polygraph sentiment among other professionals.

Anywhooo, I'll weigh in against ASTM getting into the polygraph proficiency certification business until I'm convinced to do otherwise. I do not believe this is the answer to our needs, and I am not in favor of pretending to have answers when we do not have them.

As always,

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-21-2008 11:14 AM            

quote:

---

Barry, I also favor proficency testing, but when you say proficency testing is coming, are you predicting mandatory or voluntary testing and certification? And do you see this at the federal or state level? And how can this come about when so many states do not even license?

---

It's part of the APA's Strategic Plan, which has already been approved, so it's coming.

As far as ASTM goes, I've already said I don't think they can pull off that limited portion, but I think it's worth exploring and getting feedback. (None of this is going to make it to them, so respond to the survey.)

We know know people have to know how to run good tests, and proficiency testing for those portions is a monumental, albeit not impossible, task. They also have to know how to score what they end up with, and ASTM is only looking at scoring proficiency right now. That's one little tiny sliver of the equation. Personally, I'd prefer an outside party - outside of the APA who requires such testing (and sets the standards) to be in compliance with its paired testing model policy - do that type of testing. At this point though, Ray is correct in that data is a problem. Twice a year is unrealistic too.

How many bad examiners are getting tests in front of attorneys (defense and prosecution)? Too many. What if they could be asked if they've ever had their scoring proficiency tested? They'd have to lie or respond in one of two ways: 1) no, in which case the question is "Why?", or 2) "Yes, and I'm not very good."

People who don't do well then know they need work, and scoring is an easy one to improve on.

posted 06-21-2008 12:24 PM            

quote:

---

How many bad examiners are getting tests in front of attorneys (defense and prosecution)? Too many. What if they could be asked if they've ever had their scoring proficiency tested? They'd have to lie or respond in one of two ways: 1) no, in which case the question is "Why?", or 2) "Yes, and I'm not very good."

---

This is still the wrong question and wrong answer to the problem.

Scoring proficiency doesn't answer the question about that single examination - the correct solution is a QA protocol. No-one cares about how I scored on my psychotherapy licensing exam. They care about the accuracy, diagnosis, and treatment plan for the individual patient who's case is in court.

In seeking to improve the professionalism among polygraph examiners, it does not make sense to set for ourselves a bunch of standard/certification requirements that are inconsistent with and go way beyond the requirements of related professions (medicine & psychology).

Proficiency testing is an important concern, and the time of initial certification and recertification if deemed necessary. Neither medicine, nor psychology requires their professional membership to re-take a licensing exam at the time of recertification.

Some skills are, of course, more perishable than others. Weapons proficiency for example, requires continual practice.

If we are going to say that polygraph scoring proficiency is a highly perishable skill - and not an objective, measurement-based, scientific approach to data analysis - then we could potentially create more criticism and concern about polygraph tests than before.

The goals and objectives of the APA should be to improve skills and professionalism, without increasing liability or professional vulnerability.

Individual cases that go to court, or some other critical decision, will always have to stand on their own merits. We have too often assigned credibility because some examiner has been at the job for so-many years, has conducted so-many thousand tests, or came from a certain school - or hasn't been on the job so long, hasn't run so many test, or came from the wrong school. These have nothing to do with data and skill, and amount to the use of personality as a metric for competence - because we can't even find agreement on the data features within a set of charts.

Proficiency testing is important. But its most important for initial certification requirements. After that, long term stability in the field will be best facilitated by improved QA protocols - so that we don't make competency judgments in response to charming or forceful personalities, strong opinions or belief systems, shiney boots, big cigars, expensive watches, and all the other penis-envy crap (including the size of one's agency) that has nothing to do with the quality of a single exam that may be used to make a decision that may affect the future of an individual.

That goal is best met by QA, not proficiency testing.

Proficiency testing is fine, but its not much. Passing a medical or psychotherapy licensing is exam is also fine, and also not much. Licensing boards know this. That is why they also require supervised field practice prior to licensure. Psychologists used to be required to complete an oral board exam also, but that practice failed a legal challenge which exposed the fact that oral exam proficiency has little if anything to do with effective psychotherapy practice.

The point is, that if we feel compelled to do something - doing just anything, because its something, is not the correct solution. We have to do what matters, and we should do only that which matters.

Someone could conceivably score a proficiency testing sample perfectly, then still have a bad day and score one test incorrectly. If proficiency testing is our regulating paradigm, that test would go to court - only to be embarrassed by the opposing counsel's expert. We would have missed it because we depended on personality (the examiner's past proficiency results) instead of the present data.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-21-2008 12:44 PM            

quote:

---

Scoring proficiency doesn't answer the question about that single examination - the correct solution is a QA protocol.

---

posted 06-21-2008 04:35 PM            

These issues are simpler than you might imagine.

Who's proficient enough to complete a QA review? Anyone whom the consumer decides or the court allows - almost. I would argue that perhaps any examiner who is fully qualified and experienced could do so. Ultimately it will be the consumer who decides who has the objectivity and the expertise to complete a competent and objective review (on behalf of the consumer), and it will be the court and/or consumer who decides whether the expert reviewer or expert examiner's credentials are acceptable.

You saw these principles in the Sinclair/Obama polygraph from Mr. Gelb, which Gordon Barland reviewed. You don't have to like Sinclair, Obama, Mr. Gelb's doctorate, or the test result. Its a good example of an attempt at a consumer driven QA request.

While Maschke may have been critical about some of the non-blind aspects of the review, I am not bothered by those concerns. Studies have rather consistently shown that original examiners scores have tended to be closer to ground truth than blind reviewers. So, there appears to be little, if anything, to be gained from a blind review. Instead there is probably something to be gained from contact between a reviewer and examiner (resolution of questions, feedback, etc.) Blindness is for research studies, not field-work. Just like ecological sampling is for research, not competency certification.

I am in favor of some involvement from the original examiner in the selection of a reviewer (in the private practice world), because that stabilizes both personality conflicts and the potential for biased reviews. If the examiner and consumer cannot select and agree on a competent and objective reviewer (perhaps the examiner feels that no-one is qualified to review his or her work, or expresses paranoia that everyone will unfairly criticize the work), the consumer already has the answer (a rogue and isolated examiner).

QA in private practice is a little different than in agencies. In an agency, QC is a department or program itself, and every test might be subject to oversight. In an agency, the referrer, standard-setter, trainer, certifier, reviewer, and consumer of test results may all be the same agency. This works because agencies have a mandate to do things this way, and the resources to do it.

In private practice, the roles of referrer, standard-setter, trainer, certifier, reviewer, and consumer are not housed in a single agency, but are distinct and separate roles. The process is driven by the consumer, according to established professional standards. In the absence of standards, anything goes (almost) - which is why standards are important. In private practice there is neither a mandate (local enforcement), nor the resources for a program, so QA goals are met again through protocols or standards that define necessaryt QA activities and principles. The process is again driven by the consumer's needs, according to the consumers resources. QA activities are best completed according to defined standards for QA activities, else we're back to the Cole-Porter problem (anything goes).

The thing that is important is that consumers become better educated about QA protocols for private practice polygraphy, and that the process be structured to avoid rubber-stamp and nit-picking QA reviews. Structure and standard protocols for QA activities can address this effectively.

The processes of continuing education requirements, coupled with the expectation for professional accountability at the level of the individual case, are the common methods, in related professions (medicine, psychology) for ensuring competent field work in private practice settings. Excessive emphasis on recertifying competency creates a distraction from purpose and increases liability. Just look at how much $ and energy our school systems are wasting with all of the increased emphasis on teacher and school accountability systems that don't seem to be working. Teachers and school are now busy proving they are teaching that all they put more time and emphasis than ever into teaching students to perform well on the standardized proficiency tests, out of fear for their own professional liability/survival, but are neglecting aspects of the more important curricula.

Professional accountability is sometimes a nice professional football to lob about, but lets be careful to get it right and not replicate the no-child-left-behind experience.

I believe that certification and ongoing recertification distracts our attention from the things we ought to be concern about, and pacifies us inaccurately.

Certification is an entry level issue only.

After that, it is the work itself and the needs of the consumer, coupled with their right to obtain a second opinion and naturally occurring oversight processes that ensures competent work and draws attention to incompetency.

As a profession, polygraph examiners are rather quick to indict each others' work as bad or invalid. I believe this is a lot less common in other professions. Sure there are marketing concerns, but the overall rate at which we tend to disagree and the lack of uniformity/standards tends to make us look silly at times. If a polygraph exam were "invalid" every time someone disagreed, there would be no "valid" polygraph anywhere.

Certification is fine, but that's what polygraph school is for.

If polygraph skills are so perishable that we need to continuously re-certify our ability to score charts, then perhaps our arcane systems are still too complicated we should be further simplifying our methods so they are more objective and less perishable.

Sorry for the rant.

I've been away, and I've missed this forum.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-22-2008 01:27 AM            

Although blind scoring charts has its use in inter rater reliability, it really does not show that an examiner is competent at conducting a proper examination that will end in a correct opinion.

It might for the field, at best, help point to good blind chart scorers that have the potential to be good QA examiners, as they have shown that they can correctly discern physiological tracing. However, there are still other qualities that need to be present.

As Ray pointed out, QA should be a more pressing requirement. However, there is still much work needed in other areas prior to this.

As for the ASTM, here are some of the other proficiency areas:

quote:

---

Metals Testing

Plastics Testing

Textiles

Engine Coolants Testing

Electrical Insulating Liquids

Aromatic Hydrocarbons

Cement & Concrete Reference Laboratory (CCRL)

NEW! Ores & Metals Environmental

---

I fail to see the connection between these tests and what we do. After all, we do conduct an “examination”.

We should look at the (big) APA, ASCLD and AAFS professional models for just that, a “professional model”.

We should, as a profession, press for national licensing, rules, and regulations to help ensure that those states that lack such things have oversight and, in turn, hopefully help create a sense of urgency for those states to do so.

As I and others have said before, we still lack a theoretical basis for what we do and the research to support it. This should be priority number one.

posted 06-23-2008 12:00 AM            

I am well aware that ASTM sets standards. To be blunt, I fail to see what relevance it has.

I have yet to have heard a forensic scientist that I have spoken to say that they were asked a question regarding ASTM standards. They have been asked about ASCLD certification in regards to their laboratory.

I agree with you that the ASTM should not be certifying or enforcing standards, only setting them. As you know, courts do not certify or enforce standards of a particular discipline but of rules of evidence.

posted 06-23-2008 11:26 AM            

I just spoke with Anne McKlindon, ASTM, and originator of the survey e-mail. She explained the following:

This basis for this survey was submitted by Marty, Don and Jack and endorsed by the ASTM E-52 Executive Committee as a means of determining interest in a self-proficiency program. THIS IS NOT A CERTIFICATION PROGRAM OF ANY SORT, FORM OR DESIGN!

As she explained, the process would include
a semi-annual review of charts to determine proficency in evaluation of test data, as anonymously compared to others in the field and against known truth cases. No-one would know your "score" or ability and the overall results are not open for review or inspection by anyone else, including other E-52 committee members.

Further, the PTP programs elsewhere discussed within ASTM are NOT CERTIFICATIONS of any type or form. In fact, she decried this statement that this survey represented any form of "certification", now or for in the future.

If anyone has any further questions about this survey or its intent, she advised you may contact her at the e-mail address on your survey, or call 610-832-9688.

Thank you,

posted 06-23-2008 01:16 PM            

now that you mention it; let's set the stage for that very misuse of semantics:

Grogan takes the proficiency examination and "scores" better than some and is considered "proficient" by ASTM standards
(Remember, this "examination" will not be proctored).

Does anyone else think that this will be his next promotional statement?

"John Grogan, Certified Polygraph Examiner, as established by the ASTM." This of course is better than simply, "John Grogan, Member -ASTM"

Turns my stomache...